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TWO-PLAYER NONZERO SUM STOPPING GAMES IN DISCRETE 

TIME 

By Eran Shmaya and Eilon Solan 1 

Tel Aviv University 

We prove that every two-player nonzero-sum stopping game in 
discrete time admits an e-equilibrium in randomized strategies for 
every e > 0. We use a stochastic variation of Ramsey's theorem, which 
enables us to reduce the problem to that of studying properties of 
e-equilibria in a simple class of stochastic games with finite state 
space. 

1. Introduction. The following optimization problem was presented by 
Dynkin (1969). Two players observe a realization of two real- valued processes 
(x n ) and (i? n ). Player 1 can stop whenever x n > 0, and player 2 can stop 
whenever x n < 0. At the first stage 9 in which one of the players stops, 
player 2 pays player 1 the amount Rq and the process terminates. If no 
player ever stops, player 2 does not pay anything. 

A strategy of player 1 is a stopping time \x that satisfies {/i = n} C {x n > 
0} for every n > 0. A strategy v of player 2 is defined analogously. The ter- 
mination stage is simply = min{/i, u}. For a given pair (/x, u) of strategies, 
denote by 

7(ji,u) = B[l {e<oo} R e ] 

the expected payoff to player 1. 

Dynkin (1969) proved that if sup n>0 \R n \ £ Li, this problem has a value v; 
that is, 

v = supinf 7(/i, u) = inf sup7(/i, v). 

fj, v v (j, 

He moreover characterized e-optimal strategies; that is, strategies /i (resp. 
u) that achieve the supremum (resp. the infimum) up to e. 
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Neveu (1975) generalized this problem by allowing both players to stop at 
every stage, and by introducing three real- valued processes (i?m )n ) , (-R{2},n) 
and (R{i,2},n)- The expected payoff to player 1 is defined by 

7(M) v ) = E [l{/i<i>}-R{l},/i + 1 {n>v}R{2},v + 1 {/x=v<oo}-R{l,2},/i]- 

Neveu (1975) then proved that this problem has a value, provided 

(a) sup n > max{| J R {1}jn |, \R {2 },n\, \R{x,2},n\} € Li and 

( b ) R{l},n = R{l,2},n<R{2},n- 

Recently Rosenberg, Solan and Vieille (2001) studied games in Neveu's 
setup, but allowed the players to use randomized stopping times: a strategy 
is a [0, l]-valued process that dictates the probability by which the player 
stops at every stage. They proved that the problem has a value, assuming 
only condition (a). 

Extensive literature provides sufficient conditions for the existence of the 
value in continuous time [see, e.g., Bismut (1977), Alario-Nazaret, Lepeltier 
and Marchal (1982), Lepeltier and Maingueneau (1984), Touzi and Vieille 
(2002) and Laraki and Solan (2002)]. Some authors have studied the diffusion 
case, see, for example, Cvitanic and Karatzas (1996). 

The nonzero-sum problem in discrete time when the payoffs have a special 
structure was studied, among others, by Mamer (1987), Morimoto (1986), 
Ohtsubo (1987, 1991), Nowak and Szajowski (1999) and Neumann, Ram- 
sey and Szajowski (2002) and the references therein. In the nonzero-sum 
case, the processes (R{i^ n ), (R{2}, n ) an d {R{i,2},n) are R 2 -valued, and the 
expected payoff to player i, i = 1,2, is 

7*0^) = E »A 1 {n<v} R {l},H + 1 {^>u}R\2},v + 1 {^=iy<oo}R\l : 2},fi]- 

The goal of each player is to maximize his or her own expected payoff. Given 
e > 0, a pair of stopping times (fi, v) is an e- equilibrium if for every pair of 
stopping times (fj,',v'), 

7 1 (^,i / )>7 1 (^,^)-£ and 7 2 (^, v) > 7 2 (/U, v') - e. 

The above-mentioned authors provided various sufficient conditions under 
which e-equilibria exist. 

In the present paper, we study two-player nonzero-sum games in discrete 
time with randomized stopping times, and we prove the existence of an e- 
equilibrium for every e > 0, under merely an integrability condition. Our 
technique is based on a stochastic variation of Ramsey's theorem. Ramsey 
(1930) proved that for every coloring of a complete infinite graph by finitely 
many colors there is a complete infinite monochromatic subgraph. Our vari- 
ation allows us to reduce the problem of the existence of an e-equilibrium 
in a general stopping game to that of studying properties of e-equilibria in 
a simple class of stochastic games with finite state space. 
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The paper is arranged as follows. In Section 2 we provide the model and 
the main result. A sketch of the proof appears in Section 3. In Section 4 
we present a stochastic variation of Ramsey's theorem. In Section 5 we 
define the notion of games played on a finite tree and we study some of 
their properties. The proof of the main result in this section is relegated 
to Section 8. In Section 6 we show how to approximate a general filtration 
between two stopping times by a sequence of finite algebras. In Section 7 
we construct an e-equilibria. We end by discussing extensions to more than 
two players in Section 9. 

Our proof uses tools both from the theory of stochastic processes (Sections 
4 and 6) and from the theory of games (Sections 5 and 8). However, no prior 
knowledge of these fields is assumed. 

2. The model and the main result. A two-player nonzero-sum stopping 
game is a 5-tuple F = (Q,A,p, R) where: 

• (Q,A,p) is a probability space; 

• T = {J : ti)ti>q is a filtration over (Q,A,p); 

• R = (R n )n>o is an ^"-adapted R 6 -valued process. The coordinates of R n 
are denoted by Rq n , i = 1,2, <j> ^ Q C {1, 2}. 

A (behavior) strategy for player 1 (resp. player 2) is a [0, l]-valued ^-adapted 
process x = (x n ) n >o [resp. y = (y n )n>o\- The interpretation is that x n (resp. y n ) 
is the probability by which player 1 (resp. player 2) stops at stage n (pro- 
vided the game is not stopped before that stage). 

Let 8 be the first stage, possibly infinite, in which at least one of the 
players stops and let ^/QC{1,2} be the set of players that stop at stage 
9 (provided < oo). The expected payoff under (x,y) is given by 

(1) 1 i (x,y)=Ex > y[ R Q,8 1 {e<oo}], 

where the expectation E x y is with respect to (w.r.t.) the distribution P x ,y 
over plays induced by (x,y), and 1 is the indicator function. 

Definition 2.1. Let F= -4,p, JF, i?) be a two-player nonzero-sum 
stopping game and let e > 0. A pair of strategies (x* ,y*) is an e-equilibrium 
if 7 1 (x*,y*) > 7 1 (x,y*) — e and j 2 (x*,y*) > j 2 (x*,y) — e for every x and y. 

The main result of the paper is the following: 

Theorem 2.2. Let F = (fl,A, p,3~, R) be a two-player nonzero-sum stop- 
ping game such that sup n>0 ||-R n ||oo G ^(p). Then for every e > 0, the game 
admits an e-equilibrium. 
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The definitions imply that for every two payoff processes R and R such 
that E[sup ngN \\R n — R n \\oo] < £> every e-equilibrium in (Q, A, p, J 7 , R) is 
a 3e-equilibrium in (0, A,p,J-, R). Hence we can assume without loss of 
generality (w.l.o.g.) that the payoff process R is uniformly bounded and 
that its range is finite. Actually, we assume that for some K E N, R n E 
{0,±^,±|,...,±f} 6 for every n E N. 

3. Sketch of the proof. In the present section we provide the main ideas 
of the proof. Let T be a stopping game. To simplify the presentation, assume 
that J- n is trivial for every n, so that the payoff process is deterministic. 
Recall that w.l.o.g. payoffs are uniformly bounded by 1. 

Given e > 0, fix a finite covering M of the space of payoffs [— 1,1] 2 by 
sets with diameter smaller than e. For every two nonnegative integers k <l, 
define the periodic game G(k, I) to be the game that starts at stage k and, if 
not stopped earlier, restarts at stage I. Formally, G(k,l) is a stopping game 
in which the terminal payoff at stage n is equal to the terminal payoff at 
stage k + (nmodl — k) in T. 

This periodic game is a simple stochastic game [see, e.g., Shapley (1953) or 
Flesch, Thvijsman and Vrieze (1996)] and is known to admit an e-equilibrium 
in periodic strategies. Assign to each pair of nonnegative integers k < I an 
element m(k, I) E M that contains the expected payoff that corresponds to 
a periodic e-equilibria of the game G(k,l). 

Thus, we assigned to each k < I a color m(k,l) E M. A consequence of 
Ramsey's theorem is that there is an increasing sequence of integers < 
k\ < hi < • ■ ■ such that m(fei, A^) = m(k n , k n +i) for every n. 

Assume first that k\ = 0. A naive candidate for a 3e-equilibrium suggests 
that between stages k n and k n+ \ , the players follow a periodic e-equilibrium 
in the game G(k n ,k n+ \) with corresponding payoff in the set m{k\,k2)- 

For this strategy pair to indeed be a 3e-equilibrium, the properties of the 
e-equilibria in periodic games must be studied. The complete solution of this 
case appears in Shmaya, Solan and Vieille (2003), who observed that in each 
periodic game G(k, I) there exists a periodic e-equilibrium that satisfies at 
least one of the following conditions: (i) Neither player ever stops, (ii) Both 
players receive nonnegative payoffs and termination occurs in each period 
with probability at least e 2 . (iii) If a player receives a negative payoff, then 
his or her opponent stops in each period with probability at least e 2 . The 
fact that at least one of these conditions holds is sufficient to prove that 
the concatenation described above is a 3e-equilibrium, with corresponding 
payoff in the convex hull of m{k\,k2)- 

If k\ > 0, choose an arbitrary m E m(ki, fcj)- Between stages and k\, the 
players follow an equilibrium in the k\ -stage game with terminal payoff m; 
that is, if no player ever stops before stage k\, the payoff is m. From stage ki 
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and on, the players follow the strategy described above. It is easy to verify 
that this strategy pair forms a 4e-equilibrium. 

When the payoff process is general, few difficulties appear. First, a peri- 
odic game is defined now by two stopping times fi\ < fi2 '■ Ml indicates the 
initial stage and \l<l indicates when the game restarts. To analyze this peri- 
odic game, we have to reduce the problem to the case where the cr-algebras 
>-7>i+i) • • • )-7> 2 are finite. This is done in Section 7. 

Second, we have to study properties of e-equilibria in these periodic 
games, so that a proper concatenation of e-equilibria in the different pe- 
riodic games generates a 4e-equilibrium in the original game. This is done 
in Section 5. 

Third, we have to generalize Ramsey's theorem to this stochastic setup. 
This is done in Section 4. 

4. A stochastic variation of Ramsey's theorem. In the present section 
we provide a stochastic variation of Ramsey's theorem. Let (£l,A,p) be a 
probability space and let T = (J- n )n>o be a filtration. For every set A C Q, 
A c = Q \ A is the complement of A. For every A, B G A, A holds on B if and 
only if p(A c nB)=0. 

Ramsey (1930) proved that for every function that attaches a color c(k, I) € 
C, where C is a finite set, to every two nonnegative integers k <l, there is 
an increasing sequence of integers ko < k± < ■ ■ ■ such that c(ko, k\) = c(ki,kj) 
for every i < j. 

We are going to attach for every nonnegative integer n and every stopping 
time r, an ^-measurable function c„ iT that is defined over the set {r > n}, 
whose range is some finite set C. We also impose a consistency requirement: 
if ri = T2 > n on an J^-measurable set F, then c„ )T1 = c n)T2 on F. Under 
these conditions, we derive a weaker conclusion than that of Ramsey's the- 
orem: for every e > there exists an increasing sequence of stopping times 
00 < #1 < • • ■ such that P(c0 Otdl = c 9u9i+1 Vi) > 1 - e. 

We now formally present the result. 

Definition 4.1. An NT function is a function that assigns to every 
integer n > and every bounded stopping time r an JP" n -measurable ran- 
dom variable (r.v.) that is defined over the set {r > n}. We say that an 
NT function / is C -valued for some set C if the r.v. /„ jT is C-valued for 
every n > and every r. 

Definition 4.2. An NT function / is T '-consistent if for every n > 0, 
every ^-measurable set F and every two bounded stopping times Ti,T2, we 
have 

n = r 2 > n on F implies f UjT1 = /„ iT2 on F. 
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When / is an NT function and a <r are two bounded stopping times, we 
denote / CT)T (u;) = f a ( u ) lT (kj)- Thus f a>T is an ^v-measurable r.v. 

The main result of this section is the following. 

Theorem 4.3. For every finite set C of colors, every C -valued T -consistent 
NT function c and every e > 0, there exists a sequence of bounded stopping 
times < 9o < 9\ < 62 < ■ • ■ such that p(cg 0i g 1 = cq x $ % = cg 2: g 3 = •••)> 1 — e. 

Comment. The natural stochastic generalization of Ramsey's theorem 
requires the stronger condition p(cg 0i g 1 = cg it $ j V0 < i < j) > 1 — e. We do 
not know whether this generalization is correct. 

The following example shows that a sequence of stopping times 0q <0± < 
02 < ■ • ■ such that p(cg 0t g 1 = ce 1 ,e 2 = •■•) = I need not exist, even without the 
boundedness condition. 

Example 4.4. Let X n be a biased random walk on the integers, let 
X = and let p{X n+1 =X n + l) = l- p(X n+1 = X n - 1) = 3/4. Let T n = 
a(Xo,Xi, . . . , X n ). Let Rq = Q and, for every n > 0, let R n = Ui<fc<n{^fc = 
— 1}. For every finite (but not necessarily bounded) stopping time r de- 
fine c n:T = Red on R n n {r > n} and c„ iT = Blue on R c n n {r > n}. Since 
p(U n >o R n ) < 1) whereas for every finite stopping time and every B £ Tg, 
one has p(\J n>0 R n \B) > 0, it follows that for every sequence 0$ < 0\ < • ■ • 
of finite stopping times p(ce 0j g 1 = Blue) > 0, whereas p{cg Q: g 1 = cg lt g 2 = ■ ■ ■ = 
Blue|ce 0i e 1 = Blue) < 1. 

We start by proving a slightly stronger version of Theorem 4.3 for \C\ = 2. 

Lemma 4.5. Let C = {Blue, Red} and let c be a C-valued T -consistent 
NT function. For every e > there exist N £ N, two sets R,B £ and a 
sequence N < tq < t\ < T2 < ■ ■ ■ of bounded stopping times, such that: 

(a) R = B C ; 

(b) p(c TOiT1 = c ri;T2 = • • • = Red\R) > 1 - e; 

(c) p(cr k>n = Blue Vk,l\B) > 1 - e. 

Proof. We claim first that for every n £ N we can find two sets R n , B n £ 
T n and a bounded stopping time a n such that: 

1. We have p(R n U B n ) > 1 - 1/2". 

2. We have {a n > n} on R n and c„ )fTn = Red on R n . 

3. For every bounded stopping time r, c„ )T = Blue on B n n {r > n}. 
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To see this, fix n G N. Call a set F G F n red if there exists a bounded 
stopping time of such that on F both op > n and c n ^ F = Red. Since c is 
^"-consistent, if F, G G F n are red, then so is F U G. Let a = sup^{p(F), F G 
F n is red}. For every A; > 1, let F& G F n be a red set such that p{Fk) > a — r- 
Let F* = Ufc>i Ffc. Observe that F* G F n and p(F*) = a. Moreover, no subset 
of F£ with positive probability is red. Let R n = F2™, , let a n be a bounded 
stopping time such that on R n , a n > n and c n ^ n = Red, and let B n = Ff . 
This concludes the proof of the claim. 

Let B = {B n i.o.j, and set R = B c . Since F,F G V„F n , N_£ N and 
there are two sets B,R G Fat such that (i) R = B c , (ii) p{B\B) > 1 — e 
and (hi) p(R\R) > 1 — e. On F, and therefore also on Fn F, both B n and 
(F n U F n ) c occur only finitely many times. By sufficiently increasing N, we 
assume w.l.o.g. that p(f] n>N R n \Rn R) > 1 — e. In particular, 



(2) p( n ^ 



R > 1 - 2e. 



Let = no < < n2 < • • • be a sequence of integers such that, for 
every fc > 0, p(F fc |F n F) > 1 - e/2 k +\ where T fc = \Jn k <n<n i+k B n . Then 
p(rifc>oFfc|F n B) > 1 — e and, therefore, 



(3) pi f| F fc F >l-2e 




We now define the sequence (rk)k>o inductively, working separately on 
R and B. Consider first the set R. Define To = N. Given define Tk+\ = 
EneN^nl^^nRnriR 011 F H Un({ T fc = n} n F n ). Since r fc and (<7„)„> are 
bounded, Tfc +1 can be extended to a bounded stopping time on R. By def- 
inition c T(hT1 = c T1)T2 = • • • = Red on R n (Dn>iV ^n); an d ^ follows from (2) 
that p(c TOjT1 = c TljT2 = • • • = Red|F) > 1 — 2e. 

Consider now the set F. Define ro = iV. Define Tk+i(w) = mm{nk <n < 
nk + i,w G B n } on F n Fc and t^+i = rik+i — 1 on B \ T^. By the definition 
of Tfc, for every fc, / G N, c TfcjT; = Blue on F n (Ofc>o ^fe) an d h follows from (3) 
that p(c Tk)Tl = Blue Mk,l\B) > 1 - 2e. □ 

Proof of Theorem 4.3. We prove the theorem by induction on \C\. 
The case \C\ = 2 follows from Lemma 4.5. Assume we have already proven 
the theorem for \C\ = r and assume \C\ = r + 1. Let Red be a color in C. 

By considering all colors other than Red as a single fictitious color and by 
applying Lemma 4.5, there exist N G N, two sets F, B G F/v and a sequence 
of stopping times N < tq < t± < • ■ • such that (i) R = B c , (ii) p(c TOiT1 = 
c ri T2 = . . . = Red|F) > 1 - e/2 and (hi) p(c TfejT; / Red \/k,l\B) > 1 -'e/2. 
We define Q{ separately on R and B. On F, we let 0* = 7*. 
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We now restrict ourselves to the space (B ,Ag,p§) with the filtration Q n = 
T Tn n B. Let c be the C-valued NT function over Q defined by c n j( g — c Tn]T(3 
for every stopping time (3 of Q, where rp = J2 n T nl{p= n } is a stopping time 
of T . Let d be the coloring that is obtained from c by swapping the color 
Red with another color in C, say Green: 

, _ j c n .$, if Cn,/3 ¥= Red, 

\ Green, if c nj( g = Red. 

Since d is a C \ {Red}-valued ^-consistent NT function, we can apply the 
induction hypothesis and obtain a sequence of stopping times < 0q < f3\ < 
fa < • • • of Q such that 



B)>l-e/2. 



( 4 ) ?(^A=^ 

By (4) and (iii) it follows that p(c j( 3 0i/ g 1 
G% = T /3i on B. Thus 

( 5 ) P( c o ,9i = c di,e 2 = 
Combining (ii) and (5) we get p(cg 0j g 1 - 



= cp u fc = ■ ■ ■ \B) > 1 — s. We define 
■■■\B)>l-e. 

- c 0i,e 2 = ' ") > 1 — e > as desired. □ 



5. Stopping games on finite trees. An important building block in our 
analysis is stopping games that are played on a finite tree. In the present 
section we define these games and study some of their properties. 



5.1. The model. 



Definition 5.1. A stopping game on a finite tree (or simply a game on 
a tree) is a tuple T = (S,Si,r,(C s ,p s ,R s ) seS \Si), where 

• (S, Si,r, (C , s ) sg 5\5' 1 ) is a tree, S is a nonempty finite set of nodes, S\QS 
is a nonempty set of leaves, r G S is the root and, for each s € S \S\, 
C s C 5 \ {r} is the nonempty set of children of s [we denote by So = S \ Si 
the set of nodes which are not leaves; for every s G S, depth(s) is the depth 
of s — the length of the path that connects the root to s] 

and for every s & So, 

• p s is a probability distribution over C s ; 

• fl s 6 R 6 is the payoff at s [the coordinates of R s are denoted (Kq s )i=i^, <^qc{i,2}]- 

A stopping game on a finite tree starts at the root and is played in stages. 
Given the current node s G So, and the sequence of nodes already visited, 
both players decide, simultaneously and independently, whether to stop or 
to continue. Let Q be the set of players that decide to stop. If Q 7^ <j>, the 
play terminates and the terminal payoff to each player i is Kq s . If Q = (ft, a 
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new node s' in C s is chosen according to p s . The process now repeats itself, 
with s' being the current node. If s' € Si, the new current node is the root r. 
Thus, players cannot stop at leaves. 

The game on the tree is essentially played in rounds. The round starts at 
the root and ends once it reaches a leaf. 

Consider the first round of the game. Let t be the stopping stage. If no 
termination occurs in the first round, t = oo. If t < oo, let s be the node (of 
depth t) in which termination occurred and let Q be the set of players that 
stop at stage t. The r.v. r l = Rq s l{t <0 o} is the payoff to player i in the first 
round. 

A stationary strategy of player 1 (resp. player 2) is a function x : So — > [0, 1] 
(resp. y : So — > [0, 1]); x(s) is the probability that player 1 stops at s. Denote 
by P x ,y the distribution over plays induced by (x,y), and denote by E Z)J/ 
the corresponding expectation operator. 

For every pair of stationary strategies (x, y) we denote by n(x, y) = ~P X y(t < 
oo) the probability that under (x,y) the game terminates in the first round 
of the game; that is, the probability that the root is visited only once along 
the play. We denote by p l (x,y) = E Xj?/ [r*], i = 1,2, the expected payoff of 
player % in a single round. Finally, we set j l (x,y) = p l {x,y) /n(x,y) (by con- 
vention, jj = 0). This is the expected payoff under (x,y). In particular, 

(6) n(x,y) x j l (x,y) =p l (x,y). 

When we want to emphasize the dependency of these variables on the 
game T, we write n(x,y;T), p l (x,y;T) and j l (x,y;T). 
Observe that for every pair of stationary strategies (x,y), 

(7) 7r(x,0)+7r(0,y)>7r(a;,y), 

where is the strategy that never stops; that is, 0(s) = for every s. 

Definition 5.2. A pair of stationary strategies (x,y) is an e- equilibrium 
of the game on a tree T if, for each pair of strategies (x',y'), 7 1 (x',y) < 
7 1 (x,y) +e and j 2 (x,y') < j 2 (x,y) + e. 

Comment. A stopping game on a finite tree T is equivalent to a re- 
cursive absorbing game, where each round of the game T corresponds to a 
single stage of the recursive absorbing game. A recursive absorbing game 
is a stochastic game with a single nonabsorbing state in which the payoff 
in nonabsorbing states is 0. Flesch, Thuijsman and Vrieze (1996) proved 
that every recursive absorbing game admits an ^-equilibrium in stationary 
strategies. This result also follows from the analysis of Vrieze and Thuijs- 
man (1989). However, there is no bound on the per-round probability of 
termination under this e-equilibrium and we need to bound this quantity. 
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5.2. Main results concerning games on trees. Throughout this section 
we fix R l ,R 2 £ R, such that at least one of them is positive, and play a 
game on a tree whose payoff's (R s ) se s satisfy the following conditions for 
every i = 1,2, every 0CQC{1,2} and every node s £ So: 

Bl. For some K £ N, s £ {0, . . . , ±f }. 

B2. We have ' 

B3. Whenever R\ 1} s = R 1 , R 2 {1}s < R 2 . 

B4. Whenever RlJ = R 2 , R\ 2} s < R 1 . 

We have already seen that condition Bl can be assumed w.l.o.g. We will 
later set R 1 to be an upper bound of {Rh\ n ), so that condition B2 can 
be assumed. The results we prove in this section are not trivial only when 
R\n s — R 1 and R 2 2 } s ' = ^ 2 ^ or some s, s' G So- As we see later, when con- 
dition B3 or B4 does not hold, a simple e-equilibrium exists. 

Assuming no player ever stops, the collection (p s )s<=s °f probability dis- 
tributions at the nodes induces a probability distribution over the set Si 
of leaves or, equivalently, over the set of branches that connect the root to 
the leaves. For each set D C So, we denote by po the probability that the 
chosen branch passes through D. For each s E S, we denote by F s the event 
that the chosen branch passes through s. 

We first bound the probability of termination in a single round when the 
e-equilibrium payoff is low for at least one player. 

Lemma 5.3. Let e > and let (x,y) be a stationary ^-equilibrium in T 
such thatj 1 (x,y) < R l — e. Then ir(0,y) > where fi\ = fJ>i(T) = p([j{F s , R 

R 1 }) is the probability that if both players never stop, the game visits a node s 
with R{iy s = R 1 in the first round. 

An analogous statement holds for player 2. 

Proof of Lemma 5.3. Consider the following strategy z of player 1: 

[1, if-R/ 1} ! s = R 1 i 
z s = \ w 

[ 0, otherwise. 

Denote by t\ and £2 the stopping stages of the two players in the first round. 
By the definition of z, and since payoffs are bounded by 1, 

p 1 (z,y) = P z , y (t 1 < mm{oo,t 2 }) x R 1 + E z>y [l t2 < tl r 1 ] 
(8) > P z , y (ti < 00) x R 1 -2P Z) y(t 2 < 00) 

= ^ 1 R 1 -2TT(0,y). 
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Since (x,y) is an | -equilibrium and since, by (7), vr(z,y) < 7r(0, y) +vr(z, 0) = 
7r(0,y) + A*i, 



Definition 5.4. Let T = (5, 5i,r, (C s ,p s ,R s ) se s ) an d ^' = (5', Si,r', (C 8 ,j/ a ,Rg) se s' ) 



be two games on trees. We say T" is a subgame of T if (i) 5" C S*, (ii) r' = r 
and (iii) for every s £ S' , C' s = C s , p' s = p s and i?' s = R s . 

In words, T' is a subgame of T if we remove all the descendants (in 
the strict sense) of several nodes from the tree (S, S±, r, (C s ) s e.So) an d keep 
all other parameters fixed. Observe that this notion is different from the 
standard definition of a subgame in game theory. 

Let T = (S,Si,r,(C s ,p s ,R s )seSo) De a game on a tree. For each subset 
D C So, we denote by To the subgame of T generated by trimming T from D 
downward. Thus, all strict descendants of nodes in D are removed. 

For every subgame T' of T and every subgame T" of T", let pt",t' = Ps"\s[ 
be the probability that the chosen branch in T passes through a leaf of T" 
strictly before it passes through a leaf of T" (here, S[ and S" are the sets of 
leaves of T' and T", resp.). 

The next proposition analyzes e-equilibria that yield a high payoff to both 
players. Since its proof is involved and independent of the rest of the paper, 
it is deferred to Section 8. 

Proposition 5.5. Let e e (0, 1/(36K 2 )) and, for i e {1,2}, let ai > 
R l — e. There exist a set D C Sq of nodes and a strategy pair (x,y) in T 
such that: 

1. No subgame of To has an e- equilibrium with corresponding payoffs in 
[a\,a\ + e] x [02, 02 + e]- 

2. Either (a) D = <j) (so that To = T) or (b) (x,y) is a 9e- equilibrium in T, 
a-i — £<Y( x iU) and -rr(x,y) > e 2 x po- 

Comment. Actually we prove that in case 2(b), for every pair {x\y') 
of strategies, 7 1 (x',y) < a\ + 8e and 7 2 (x,y') < 02 + 8e. 



p (z,y) = i (z,y) x 7r(^y) 



(9) 





Equations (8) and (9) imply that 7r(0, y) > | x p,i. □ 
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5.3. Coloring a finite tree. In the present section we provide an algo- 
rithm that for every finite tree T attaches a color c(T) and several numbers 
(Xj(T))j in the unit interval. 

A rectangle [a\,a\ + e] x [02, 02 + s] is bad if R 1 — e < a\ and R 2 — e < 02- 
It is 500c/ if a\ + e < R 1 — e or 02 + e < R 2 — e. 

Let M be a finite covering of [—1, l] 2 with (not necessarily disjoint) rect- 
angles [a\,a\ + e] x [02,(22 + e], ah of which are either good or bad. Thus, 
for every «£ [—1, l] 2 there is a rectangle m € M such that u £ m. We de- 
note by H = {h\,h2, • • • , hj} the set of bad rectangles in M and denote by 
G = {<7i,<?2, • • • ,gv} the set of good rectangles in M. 

Set C = GL) {0}. This set is composed of the set G of good rectangles 
together with another symbol 0. For every game on a tree T consider the 
following procedure which attaches an element c£C to T: 

• Set T(°) = T. 

• For 1 < j < J apply Proposition 5.5 to T( J_1 ) and the bad rectangle 
hj = [<ij,i,o,j,i + e] x [oj-,2, Oj,2 + e] to obtain a subgame T^> of rC? -1 ) and 
strategies (xq) ,yy ) in T( J ) such that: 

1. No subgame of has an e-equilibrium with corresponding payoffs 
in hj. 

2. Either T^' = T^~ l > or the following three conditions hold: 

(a) For i e {1,2}, a jti - e < f(x%\y$) 

(b) For every pair (x',y'), 7 1 (x',y^) < a J; i + 8e and ^y 2 (x^\y') < 
aj,2 + 8e. 

(c) We have 7r(xj) ) > e 2 x p T u) T u-i), where p T u) T u-i) is the 
probability that a randomly chosen branch passes through a leaf 
of T( J ) which is not a leaf of T^~ 1 ^ (see Section 5.1). 

• If X^) is trivial (i.e., the only node is the root), set c(T) = 0; other- 
wise choose a stationary |-equilibrium (x^°\y^) of T^. Since T^ J ' is 
a subgame of each j = 1, . . . , J, since no subgame of has an e- 
equilibrium with corresponding payoffs in hj and since every |-equilibrium 
is also an ^-equilibrium, the corresponding |-equilibrium payoff lies in 
some good rectangle g € G. Set c(T) = g. 

The strategies (xjt\y^), as given by Proposition 5.5, are strategies in T^~ x \ 
We consider them as strategies in T by letting them continue from the leaves 
of T^' -1 ) downward. 

We also define, for every j £ J, 

so that n(x!p,y!fl) > e 2 x Xj(T). 
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6. Representative approximations. Theorem 4.3 enables us to reduce 
the analysis to finite-stage games: games that start at some stage n and 
terminate at stage r, where r is some bounded stopping time. Since the state 
space Q, is arbitrary, while our game-theoretic tools allow us to analyze only 
games that are defined over a finite state space, we need to approximate 
each T n by a finite cr-algebra. 

Roughly, our goal here is to define a consistent NT function T = (T n T ) 
that assigns for every n > 0, every bounded stopping time r and every w6S], 
a game on a tree T njT (o;) that approximates the finite-stage game that is 
played between stages n and r in a desirable way. 

This is done in two stages. First, we define, for every k > n, a finite 
partition Qk of Q, such that (Qk)k>n contains all the information relevant 
to the players between stages n and r. Second, given the sequence of finite 
partitions, we define the games on a tree (T nir (aj)) wg ^. 

Throughout this section we fix a stopping game T and £ > 0. Denote 8 n — 
e 2 /2 n + 2 for each n > 0. Set A n = £ fc > n 6 k = e 2 /2 n+1 , so that £ n >o A n = e 2 . 

6.1. Partial games. In the present section we consider the partial game 
that is played between stages t\ and r 2 , where t\ < r 2 are two bounded 
stopping times, and we define and study the notion of approximating games 
on a tree. 

Definition 6.1. Let t\ < t 2 be two bounded stopping times. An [T ,t\,t 2 )- 
strategy is a sequence x = (x^) of random variables such that for every k > 0, 
(i) Xfc : {ti < k < T2} — ► [0, 1] and (ii) x k is ^.-measurable. 

Thus, an (J 7 , n, T2)-strategy prescribes to the player what to play between 
stages n and T2 (excluded). If = To < t\ < ■ ■ ■ is an increasing sequence of 
bounded stopping times and if, for each I > 0, x l is an (T, n, r/ + i)-strategy, 
we can naturally define a strategy x in the stopping game simply by concate- 
nating the strategies (a;')/>o- Similarly, any strategy x = (xk)k>o m the stop- 
ping game naturally defines an (J 7 , ti, T2)-strategy for every pair of bounded 
stopping times r± < T2 by considering the proper restriction of x. 

When x and y are two (J 7 , t\ , T2) -strategies, we define by ir(x, y; F,t±,T2) = 
^ > x,y[i-{T 1 <e<T 2 }\^ : r 1 ] the conditional probability under (x,y) that the game 
that starts at stage t\ ends before stage T2, and define by 

p(x,y;-F,Ti,T 2 ) =~E x>y [l {T1 < e<T2 }R Qi 6\-T T - L ] 

the corresponding expected payoff. We define 

j{x,y;J r ,Ti,T 2 ) = —, r. 

7r(x,y;J r ,T 1 ,T 2 ) 

These are T TX -measurable r.v.'s. 
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Definition 6.2. Let T\ < T2 be two bounded stopping times. A 5 ap- 
proximation of T between t\ and T2 is a pair ((Gk),(<lG,k)) such that for 
every k > 0: 

1. ^fc is an -measurable finite partition of {n < A; < T2}; 

2. is (^-measurable; 

3. t\ and T2 are measurable w.r.t. Q; that is, for every k > 0, {ti = fe} and 
{r2 = fc} are unions of atoms in Qk'i 

4. any atom G of such that k < T2 on G is a union of some atoms 
in <7fc+i; 

5. for every atom G of Gki qa,k is a probability distribution over the atoms 
of Gk+i that are contained in G; 

6- EG'Gg fe+ i iP^'l-^ftJM ~ QG,ki G ')\ < S k for every atom G of £ fc and 
almost every u> G G. 

We identify with the finite cr-algebra generated by and we denote 

5 = (Gk)k>0- 

With every 5 approximation (Q, (<7G,fc)) of T between t\ and r 2 and every 
atom G of £/ n we can attach a game on a tree. 

• The root is G. 

• The nodes are all nonempty atoms F of (Gk) such that (a) FCG and 
(b) if F G C/ fe , then t> k on F. 

• The leaves are all atoms -F G Ufe>n^fc where there is equality in (b). 

• The payoff is given by {Rk) T1 <k<T 2 - 

• The children of each atom F in Gk are all atoms F' in Gk+i which are 
subsets of F. 

• The transition from any node F in Gk is given by qFk- 

We denote this game on a tree by T(ti,T2;G, (qG,k),G). 

Suppose that for every atom G of Q Tl , xq is a strategy in the game 
T(Ti,T2]Q,(qG,k),G). The collection of those strategies naturally defines a 
(J r ,ri,r 2 )-strategy. 

Similarly, if x = (xk) is an (J 7 , ri,Ta) -strategy such that for every k, Xk is 
(^-measurable, then x naturally defines a strategy in T(Ti,T2]G,(qG,k),G) 
for every atom G of £/ Tl . 

Let (x, y) be a pair of (J 7 , n, r 2 ) -strategies such that for every fc, and y& 
are ^-measurable. We denote by tt(x, y,Ti,T2,G, (.<lG,k)) the probability of 
termination in one round of the game on a tree T(t±,T2;G, (lG,k),G) under 
the strategies (x,y), by p(x,y;ri,T2,G,{qG,k)) the expected payoff in one 
round, and by 



r/{x,y;n,T 2 ,G, (qG,k)) 



p{x,y\Ti,T 2 ,G,{qG,k)) 
n(x,y;n,T2,G,(qG,k)) 
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the expected payoff in the game. These three functions are Q T1 -measurable. 

The following lemma provides estimates for the difference between the 
expected payoff and the expected probability of termination, when the fil- 
tration is changed. Its proof is omitted. 

Lemma 6.3. Letr\ < t 2 be two bounded stopping times and let (Q, (qc,k)) 
be a 5 approximation of T between t\ and t 2 . Let x and y be a pair of 
(G,t±, t 2 ) -strategies. Then: 

1. \p l (x,y;J r ,T 1 ,T 2 ) - p l (x,y,T 1 ,T 2 ,g,(q G ,k))\ < A T1 for each i = 1,2. 

2. \TT(x,y;J r ,T 1 ,T 2 ) -TT(x,y,T 1 ,T 2 ,G,(q G ,k))\ < A T1 . 

The following lemma states that if (Q, is a 5 approximation of T 

between 77 and t 2 , and if the opponent plays a (Q, 77, t 2 ) -strategy, then the 
player does not lose much by considering only (Q ,t\,t 2 ) -strategies [rather 
than (T, 77 , t 2 ) -strategies] . 

Lemma 6.4. Let 77 < t 2 be two bounded stopping times and let (Q, (qc,k)) 
be a 5 approximation ofT between 77 and t 2 . Let x be a (Q ,t\,t 2 )- strategy 
for player 1 and set 7 = esssup{7 2 (x, y; Q, t±, t 2 ), y is a {Q ,t\,t 2 )- strategy] . 
Then, for every ( J 7 , t\,t 2 )- strategy y, 

p 2 (x,y; F,ti,t 2 ) < 7 x ir(x,y; F,t 1 ,t 2 ) + A Tl a.e. 

Proof. For H = T, g, let 

a(H) = esssup{p 2 (x,y;H,T\,T 2 ) 

+ 7 x (1 - ir(x,y;7i,Ti,T 2 )),y is a (H, n, r 2 )-strategy}. 

When player 2 is restricted to use (7i,Ti,T2)-strategies, a(H) is his or her 
best possible payoff in the game that starts at stage 77 and, if no player 
stops before stage t 2 , terminates with payoff 7. From the definition of 7 it 
follows that ct(Q) < 7. Define recursively 



(10) a(k,H) 



7, k > t 2 

max{z fe x R 2 {1) k + (1 - x k ) x E(a(/c + l,H)\H k ), 

Xk X R{i 2 } k + (1 — %k) X -^{2},fc}j 

k<T 2 . 



The first term of the maximization in the second line corresponds to player 2 
not stopping at stage k, while the second term corresponds to player 2 stop- 
ping at that stage. Plainly a(TC) = 0(17, H). 

Since x k and i?| are ^-measurable and since (Q, (qG,k)) is a 5 approxima- 
tion of r between 77 and t 2 , it follows by induction that a(k,J T ) < a(k,Q) + 
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J2j^=k^j f° r every k > 0. In particular, a(!F) = a{j\,F) < a(ri,Q) + A T1 < 
7 + A T1 . It follows that for every (T, t±, r 2 )-strategy y, p 2 (x, y; J 7 , T\, t 2 ) + 
7 x (1 - ix(x,y\T, ri,r 2 )) < 7 + A T1 , which implies p 2 (x,y;J : ', ti,t 2 ) < 7 x 
ir(x,y;J r ,Ti,T 2 )+A Tl . □ 

6.2. Approximating the filtration T . Our main result in this section is 
the following: 

Theorem 6.5. Let T be a two-player nonzero-sum stopping game. There 
is a consistent NT function that assigns to every n > and every bounded 
stopping time r a 5 approximation of T between n and r . 

The proof of this result, though quite technical, is intuitive. We start at 
stage r and we proceed backward until we reach stage n. For every stage 
n < k < r, we define a finite partition of Q. Roughly, the partition at 
stage k is defined in such a way that (i) payoffs at stage k are measurable 
w.r.t. Qk and (ii) the conditional probability to reach any atom G £ Gk+i 
does not vary by much over each atom of Qk- The construction is slightly 
complicated since it should be consistent. 

Proof of Theorem 6.5. For every i > and every n £ N, choose 
once and for all a partition of the (n — l)-dimensional simplex {r £ 
R n : Y^j=i r j = 1) r j — V j} such that the diameter of each element in Bf is 
less than 5i in the norm || • ||i. For each B £ Bf choose an element qs £ B. 

Fix a total order on the collection of subsets of 0. This enables us to 
identify, for every partition Q of VL to n atoms, the space R n with the 
space , simply by identifying the ith coordinate of R n with the ith atom 
of Q according to the total order. 

Fix a nonnegative integer n > and a bounded stopping time r. Let M be 
a bound on r. We first define a sequence of finite partitions (Qk) recursively 
from M down to n. We say that 00 ~m if and only if (i) t(cj) = t(uj') = M 
and (ii) Rm(u) = Rm{oj')- We let Qm be the partition of ft induced by the 
«M relation. Then Qm C Tm- 

Suppose we have already defined a finite partition Q m +i C F m -\-\. We say 
that U ~ m uj' if and only if one of the following two compound conditions is 
satisfied: 

• (i) We have t(lo) = t(oj') = m and (ii) R m (uj) = R m (u>'). 

• (i) We have t(oj) = t(uj') < m, (ii) R m (uj) = R m (uj') and (iii) the two prob- 
ability distributions (P(G\J r m )(uj)) Gs g m+i and (P(G\J r m )(u>')) G& g m+i have 
the same support, and their restrictions to their support lie in the same 
element of B^, where c = \{G £ Q m +i '■ P(G| J r m )(a;) > 0}| is the number 
of atoms of Q m +i in the common support. 
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qG,m( F ) 



We let Q m be the partition of f2 derived by the ~ m relation. 

Define for every m and every atom G of Q m a probability distribu- 
tion gc,m over the atoms of G m +i that are contained in G simply by qG,m = 
where B is the atom of B m that contains all probability distributions 
(P(G\J r m )(u>)) G£ g m+i for uj G G, and c is defined as above. 

The sequence (Q m ) satisfies the following properties: 

• We have Q m C J^" m . 

• We have that R m is £/ m -measurable. 

• For every atom G of Q m , Y,G'QG,G'eg m+ i 

QG,m (G')\ < 

The pair ((Gm), (<7G,m)) is not a 5 approximation of T between n and t, 
since (G m ) is not increasing (see condition 4 in Definition 6.2). We rem- 
edy this flaw by taking upward intersections. For every m > n let Q m = 
Vn<fc<m *3k be the finite a-algebra spanned by Q n , . . . , Q m . Let G be an atom 
of Q m . Then G C G for some atom G of ^ m . We define a probability distri- 
bution qc,m over the atoms of Q m +\ by 

' 2G, m (G")> if F = G n G' for some atom G' of g m+ i, 

0, otherwise. 

It can be verified that ((Gm),(<lG,m)) is a 5 approximation of T between 
n and r, and that each step in the construction is consistent. □ 

Theorem 6.5, together with the discussion in Section 6.1 that relates 5 ap- 
proximation to games on trees and Lemmas 6.3 and 6.4, yields the following 
corollary: 

Corollary 6.6. Let T be a two-player nonzero-sum stopping game. 
There is a consistent NT function T = (T nT ) that assigns for every n > 0, 
every bounded stopping time r and every oj G Q, a game on a tree T n ^ T (u>) 
such that the following hold for every n and every r : 

1. For every pair of strategies (x,y) inT n<T and each i = 1,2, \p l (x,y; !F, n,r)- 
p l (x,y;T UjT )\ < A n . 

2. For every pair of strategies (x,y) inT n ^ T , \ir(x,y;J r ,n,T) — 7r(x,y;T n ^ T )\ < 
A n . 

3. If (x,y) is an e-equilibrium in T n ^ T (uj) with expected payoff in [a±,ai + 
e] x [a,2, a2 + e], then: 

(i) For every strategy y' of player 2, p 2 (x,y';J r ,n,T) < (02 + 2e)Tr(x,y'; 
T,n,r) + A n . 

(ii) For every strategy x' of player 1, p 1 (x l \y\T \n,r) < (a\ + 2e)tt(x' , y; 
F,n,r) + A n . 

[In the preceding list of inequalities, we identify strategies in T n<T with 
the corresponding (J 7 , n, r)-strategies in I\] 
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7. Constructing an e-equilibrium. In the present section we use all the 

tools we have developed so far to construct an e-equilibrium. 

7.1. A sufficient condition for existence of an equilibrium. Here we pro- 
vide a sufficient condition for the existence of an equilibrium for games that 
satisfy the following conditions: 

Al. There exists K £ N such that for every n > 0, R n £ {0, ±-|, . . . , ±§ } 6 . 
A2. For every n > 0, R 1 := limsup n ^ oc ffi^i n is constant and i^ru n < R 1 ■ 
A3. For every n>0,R 2 := limsup n ^ oc i?? 2 i n is constant and R{ 2 } n — ^ ' 
A4. Whenever R\ 1} n = R 1 , R 2 {1}n <R 2 . 
A5. Whenever RlJ = R 2 , RL, < R 1 . 

Lemma 7.1. Let e > be given. Let V be a stopping game that satisfies 
conditions A1-A5. Suppose that there is a payoff vector (01,02), an increas- 
ing sequence of stopping times = To < t\ < ■ ■ ■ and, for every k > 0, a pair 
of (J 7 , T\,T2)- strategies (xk,yk) such that the following hold for every k > 0: 

Dl. For i = 1,2, p l (x k , y k ; J 7 , r k , r k +i) > (en - e)vr(x fc , y k ; T, r fe , r k+1 ) - A Tfc . 
D2. For every (J 7 , T k ,T k+ i) -strategy x' k , p 1 (x' k ,y k ;J r ,T k ,T k+1 ) < (ai+e)ir(x' k , 

yk',F,T k ,T k+1 ) + A Tfe . 

D3. For every (J 7 , T k ,r k+1 )- strategy y' k , p 2 (x k ,y' k ;J r ,T k ,T k+1 ) < (a 2 + e)7r(x k , 

y k ,^,T k ,T k+1 ) + A Tfc . 
D4. Almost surely, Y lk >o 7r (x k ,y k ;J 7 ,T k ,T k+ i) = 00. 
D5. Ifa 1 <R 1 -e, then J2k>o ^(O^ Vk\ T k, r k+ i) = 00 a.s. 
D6. Ifa 2 <R 2 -£, then Y, k >Q^(x k ^; ^ ,T k ,T k+ i) = 00 a.s. 

Then the game admits an 8e- equilibrium. 

In words, conditions D1-D3 roughly say that each (x k ,y k ) is an e-equilibrium 
with payoffs close to (01,02). Condition D4 says that the concatenation of 
all the (x k ,y k ) } s stops with probability 1, while conditions D5 and D6 say 
that if one player receives a low payoff, then his or her opponent stops with 
probability 1 under the concatenation. The proof of Lemma 7.1 is standard. 

Proof of Lemma 7.1. Denote by x (resp. y) the strategy in T that is 
the concatenation of the strategies (x k ) k >o [resp. (y k ) k >o\- 

By D4, P x , y (0 < 00) = 1. By D5, if ai < R 1 - e, then P ,y(9 < 00) = 1. 
By D6, if a 2 < R 2 — e, then P Xj o(# < 00) = 1. Therefore, there is L G N such 
that 



(11) 



Px,y(0 < t l ) > 1 - e, 
then Po )3/ (0 < tl) > 1 - e, 

then P xfi (6 < t l ) > 1 - e. 



(12) 



if ai < R 1 — e, 
if a 2 < R 2 — e, 
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We now define a pair of strategies (x* ,y*), which is a slight augmentation 
of (x,y): If ai < i? 1 — e, let y* coincide with y. If a\ > R 1 — e, let y* be 
the strategy that coincides with y up to stage tl, and from that stage 
onward, stops with probability e whenever R 2 ^} n = ^ (and with probability 

otherwise). The strategy x* is defined analogously. 

We argue that (x*,y*) is an 8e-equilibrium. We only prove that player 1 
cannot profit more than 8e by deviating. 

By summing Dl for i = 1 over k > 0, and since P Xty (0 < oo) = 1 and 
J2T=i ^r k < £, we obtain 

(13) 1 1 (x,y)>a l -2e. 

Since (x*,y*) coincides with (x,y) up to stage tl and by (11), 

l\x\y*)> 1 \x,y)-2e>a l -Ae. 

Let x' be any strategy of player 1 and, for every k > 0, let x' k be the 
(J r ,rfc,rj fc+ i)-strategy induced by x' . 

By summing D2 over k > and since 2~2kLi A Tfe < £> we obtain 

(14) 7 V, V) < (oi + eJP^lfl < +oo) + e. 

If ai < i? 1 - e, then by D5, P x >, y (6 < +oo) > P 0) y(8 < +oo) = 1. Therefore, 

J 1 (x',y)<a 1 + 2e. 
Since y* coincides with y up to stage tl and by (12), 

7 1 (x', y*)<l 1 (x',y) + 2e<a 1 + 4e<^ 1 (x* ,y*) + 8e. 

We now consider the case ai > i? 1 — e. By summing D2 over k = 0, . . . , L — 

1 and since YlkLi A Tfc < £> we obtain 

(15) E^jR^l^^y] < (ai + ejP^fl < r L ) + e. 
By the definition of y* after stage tl, it follows that 

7 V,y*) < E^[it^l {e<rL} ] + (i? 1 + e)P x ,J9 > r L ). 
By (15) and since R 1 < a± + e, 

7 1 (^ / ,y*)<ai+2e<7 1 (^,2/*) + 6e, 

as desired. □ 

As the following example shows, adding a threat of punishment might be 
necessary. 
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Example 7.2. Consider a game with deterministic payoffs: R{i} : n = 
(—1,2), R{2}, n = ( — 2,1) and Rnp\,n = (0) — 3). We first argue that all e- 
equilibrium payoffs are close to (—1,2). 

Given a strategy x of player 1, player 2 can always wait until the prob- 
ability of stopping under x is exhausted and then stop. Therefore, in any 
e-equilibrium, the probability of stopping is at least 1 — e, and the cor- 
responding payoff is close to the convex hull of (—1,2) and (—2,1). Since 
player 1 can always guarantee —1 by stopping at the first stage, the claim 
follows. 

However, in every e-equilibrium (x,y), we must have Po, y (# < oo) > 1/2, 
otherwise player 1 receives more than —1 by never stopping. 

Thus, an e-equilibrium has the following structure, for some integer N. 
Player 1 stops with probability at least 1 — e before stage N and with prob- 
ability at most e after that stage; player 2 stops with probability at most 
e before stage ./V and with probability at least 1/2 after that stage. The 
strategy of player 2 serves as a threat of punishment: if player 1 does not 
stop before stage N he or she will be punished in subsequent stages. 

7.2. Proof of Theorem 2.2. Define R 1 = limsup n ^ 00 R{iy n and R 2 = 
limsup^^oo -R| 2 j n . These are the maximal payoffs each player can guarantee 
by stopping when the opponent always continues. Recall that we assumed 
that the range 1Z of the payoff process is finite, so that p-a.s. there exist 
infinitely many n's such that R^iy n = R 1 and infinitely many re's such that 

K {2},n ~ K ■ 

We fix throughout e > sufficiently small. 

By Corollary 6.6 there is a consistent NT function (T n>T ) that assigns for 
every n > 0, every bounded stopping time r, and every cj € fi, a game on a 
tree that satisfies several desirable properties. 

Define several subsets of f2 as 

A_ = {R 1 < and R 2 < 0} 

and, for every r G 1Z, 

A rA = {(R 1 , R 2 ) = r} n I limsup R 2 n n > R 2 \ , 

\ pi pi ) 

{l},n 

A rt2 = {(R\R 2 ) = r}n \ limsup R} 2 y n >R 1 \, 

n— >oo 1 J ' 
k p2 _p2 ) 



p2 r>Z 

H {2} _-H 



A r>3 = {{R\R 2 ) = r} n (A- U4U A 2 ) c . 
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Recall that for every two payoff processes R and R such that E[sup ngN \\R n — 
Rn\\oo] < e -> every e-equilibrium in (Q,,A,p, J 7 , R) is a 3e-equilibrium in (£l,A,p,F, 

We are now going to further partition the sets (^L.^),.^. In Section 5.3 
we attached to each game on a graph T a color c(T) (from a finite set of 
possible colors) and J numbers in the unit interval (Aj(T))^ =1 . 

By Theorem 4.3 there is an increasing sequence of stopping times < 
ro < Ti < • • • such that 

P( C (T TOiT1 ) = c(T Tk , Tk+1 ) Vfc G N) > 1 - e. 

Define the subsets of ^4 r ,3, for every j = 1, . . . , J, 

U>o J 
and, for each good rectangle g £ G, 

B r ,3;g = Ar >3 D {c(T TfeiTfc+1 ) = g V fc}. 

Let be the collection of sets A_, (A-,i)j (^,2), (^4r,3 ; j), (B r ;3- g )- By 
properly modifying the payoff process on a set with measure smaller than e, 
and by possibly dropping the first few stopping times in the sequence (r^) 
and renaming the remaining stopping times, we can assume w.l.o.g. that 
(i) all the sets in V* are J> -measurable, (ii) i^ki k < R 1 and i?| 2 j k < R 2 

for every k > r , and (iii) E/=i Efc>o A j( T r fe ,r fc+1 ) < e on U^G^g- 

As the sets in V* are not necessarily disjoint, we let V be a finite partition 

of 17 into JT T0 -measurable sets such that for every atom B of V and every 

set 5* in V*, either B <Z B* or B n B* = 0. 

For every Jv -measurable set -B define the stopping game that is restricted 

to B and starts at stage r : r B)To = (B,Ab,P\b, {^r +k)k>0, (R TO +k)k>o), 

where Ab is the cr-algebra over B induced by A and p\B is the probability 

distribution p conditioned on B. 
The following lemma is standard. 

Lemma 7.3. If for every atom B of V with P(B) > 0, the game Tb,to 
admits an e-equilibrium, then the game T admits a 3e- equilibrium. 

Proof. It is well known that any finite-stage game admits a 0-equilibrium 
[one can, e.g., adapt the proof for the zero-sum case given in Rosenberg, 
Solan and Vieille (2001), Proposition 3.1]. Since ro is bounded, and since we 
changed payoffs on a set of measure at most e, the following strategy pair is 
a 3e-equilibrium: 

• If the game has not terminated by stage ro, play from that stage on an 
e-equilibrium in r_B ro , where B is the atom of V that contains u>. Denote 
by 7 the F T0 -measurable payoff that corresponds to this strategy. 
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• Until state tq , play a O-equilibrium in the game that terminates at stage To , 
if no player stops before that stage, with terminal payoff 7. □ 

Thus, it remains to show that for every atom B of V, the game Tb,t 
admits a Ce-equilibrium for some C > 0. We verify this in the next sections. 
For convenience, we assume w.l.o.g. that tq = 0. 

7.2.1. B C j4_. The game Ts,n admits a O-equilibrium: The strategy 
pair where no player ever stops is a O-equilibrium. Indeed, under this strategy 
the expected payoff for both players is 0, while any player % who stops at 
some stage k receives R 1 ^ k < R l < 0. 

7.2.2. (i) B C A rA and r 1 > or (ii) B C A r . j2 and r 2 > 0. We deal only 
with case (i). The following strategy pair is a 2e-equilibrium: Player 2 never 
stops, while player 1 stops with probability e at every stage k such that 
R\n k = = rl an d Rfiy k — ^ ne expected payoff to player 1 is r , 
while the expected payoff to player 2 is at least r 2 . Since r 1 > 0, player 1 
cannot profit by deviating, while if player 2 stops at some stage k, his or her 
expected payoff is bounded by (1 — e)R 2 2 y k + e < (1 — e)r 2 + e < r 2 + 2e. 

7.2.3. (i) i? C A r> i and r 1 < or (ii) 5 C A r ^ and r 2 < 0. We deal only 
with case (i). If r 2 < 0, then B C so that by Section 7.2.1 r# !n admits a 
0-equilibrium. Assume then that r 2 > 0. If limsup n ^ oo J j2 ^ =R 2 R\ 2 } n — 

i? , then i? C A r | 2 }, so that by Section 7.2.2 the game Tb,u admits a 
2e-equilibrium. Assume then that lirnsupyj^^ R 2^ =R 2 i?j 2 | n < R 1 . Under 
these assumptions, the following strategy pair is a 3e-equilibrium: Player 1 
stops at every stage k such that i?}^ k = r 1 and R 2 ^ k > r 2 with proba- 
bility e, while player 2 stops at every stage k > N such that i?j 2 j fc < r 1 
with probability e, where A r is sufficiently large so that under this strategy, 
player 1 stops with probability at least 1 — e prior to stage N. 

In the last two cases we use the coloring procedure presented in Section 5.3 
and the sufficient condition that appears in Lemma 7.1. 

7.2.4. B C A r ^.j. Recall that on A r ^-j, 

( 16 ) H A i( T r fe ,r fc + 1 ) = 00. 

Denote by T^\rk,Tk+i) the tree that was generated in round j of the 
coloring procedure in Section 5.3 and denote by (x^^(r/ c ,rfc_|_i),?/ ( '^(rfc,rfc +1 )) 
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the e-equilibrium with payoff in a bad rectangle [a\,a\ + e\ x [d2>«2 + £■]• By 
the definition of Xj , 

(17) vr(x^(r fc ,r fc+1 ),^)(T fc ,r fc+1 );r TfciTfc+1 ) > e 2 x Aj(T rfc)77 , +1 ). 

We apply Lemma 7.1 with 2e, {r k } and {(x^ (7^,7^+1), 2/^(7^,7^+1))}. 
Since the rectangle is bad, a\> R l — e and a 2 > i? 2 — e, so that conditions 
D5 and D6 trivially hold. 

Condition D4 holds by (16), (17) and Corollary 6.6.2. Condition Dl fol- 
lows from the choice of {x^'(T k ,T k+ i),y^(T k ,T k+ i}) an d by Corollary 6.6. 
Condition D2 follows from Corollary 6.6.3. The proof that condition D3 
holds is analogous. The desired result follows by Lemma 7.1. 

7.2.5. B C B r< 3-g. Recall that on B r ^ ]g , c(T(rk,Tk+i)) = g for every k 
and E/=iEfc>o ^j( T r k ,T k+1 ) < £• Since g = [a\ , 01 + e] X [a 2 ,a 2 + e] is a good 
rectangle, ai < R 1 — 2e or a 2 < i? 2 — 2e. We assume w.l.o.g. that a\ < i? 1 — 2e. 
Let (x(°)(r fc ,r fc+ i),?/( )(T fc ,r fc+ i)) be an §-equilibrium in T( J )(r fc ,r fc+1 ) with 
payoff in g. Since a± < R 1 — 2e, Lemma 5.3 implies that 

(18) e J 

> gMi(r(r fe ,7fe+i)) - X) A i( ^ ^:T fc+ l)■ 
j=l 

We apply Lemma 7.1, with 2e, {r fc } and {(a;(°) (r fc , r fc +i), y (0) (r fe , Tfc+i))}- 
Condition Dl holds by the choice of (x(°)(7£;,7"fc+i), J/^( r jfc) /r fe+i)) an d by 
Corollary 6.6. Conditions D2 and D3 hold as in the previous case. 
We now prove that condition D5 holds. By (18) and Corollary 6.6.2, 

e J 
n(0,y( o >(T k ,T k+1 );J r , T k ,T k+ i) > -fj,x(T(T k ,T k+1 )) -^2Xj(T Tk ,r k+1 ) - A Tfe . 

Since -R}i} jn = -R 1 ho. and since X)j=i Efc>o ^j( r r fc ,r fc+1 ) < £ on B r ^. g , it fol- 
lows that condition D5 holds. 

Condition D6 follows by an analog argument. Condition D4 follows from 
conditions D5 and D6, and since a± < R 1 — 2e or a 2 < R 2 — 2e. 

8. Proof of Proposition 5.5. This section is devoted to the proof of 
Proposition 5.5. We start by defining some new concepts for games on finite 
trees and by studying some of their properties. The proof itself appears in 
Section 8.5. 

Throughout the section we fix a stopping game on a tree that satisfies 
conditions B1-B4. 
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8.1. Union of strategies. Given n stationary strategies X\,X2, • ■ • , x n , we 
define their union x by x(s) = 1 — Ili<fc<n(l ~~ x k(s))- The probability that 
the union strategy continues at each node is the probability that all of its 
components continue. We denote x = x\ + X2 + • • ■ +x n . Given n pairs of 
stationary strategies a k = (x k ,y k ), 1 < k < n, we denote by a\ + • • ■ + a n 
the stationary strategy pair (x,y) that is defined by x = x\ + ■ • • + x n and 

y = y\ + ■■■ +y n - 

Consider n copies of the game that are played simultaneously, such that 
the choice of a new node is the same across the copies; that is, all copies 
that have not terminated at stage t are at the same node. Nevertheless, the 
lotteries made by the players concerning the decision whether to stop are 
independent. Let a k = (x k ,y k ), 1 < k < n, be the stationary strategy pair 
used in copy k and let a = ct\ + • • • + a n . 

We consider the first round of the game. Let tf. be the stopping stage in 
copy k, let Sk be the node in which termination occurred, let Qk be the set 
of players that stop at stage tk and let r\ = R % Q k Sfe l{ ifc<00 j. be the payoff in 
the first round. Set 

TTfc =ir(x k ,y k ) =P(t k < oo) and p\ = p z (x k , y k ) = E[rjy . 

Let t, r, p and ir be the analog quantities w.r.t. a: Denoting k* the index of 
a copy that stops first (so that t k * = minfc = i ) ... )n £fc), we have t = t k *, s = s k *, 
Q = Ufc:t fe =t fe « Qk and r % = R l Q s l t<+00 . Moreover, p l = p l (x,y) = ~E[r l ] and 
7r = 7r(x, y) = P(t < oo). 

Let 7^ = "f(x k ,y k ) be the expected payoff under a k = (x k ,y k ) and let 
7 = j(x,y) be the corresponding quantity under a. 

The following lemma follows from the independence of the plays given the 
branch. Recall that F s is the event that the randomly chosen branch passes 
through s. 

Lemma 8.1. Let s £ So be a node of depth j . Then, for every 1 < k, I < n, 
I ^ k, the event {t k < j} and the random variable tfcl{ ifc <j} are independent 
of ti given F s . 

Lemma 8.2. Let N = J2k=i l{t fe <oo} ^ e the number of copies that termi- 
nate in the first round. Then: 

1. We have ££=1^- - E[7V1 {7V > 2} ] < vr < £fc=i^- 

2. We have £fe=i p\ - B[(N + 1)1 {N > 2} ] < p l < Efc=i A + E[(JV + l)l {7 v>2}] 
for each player i G {1,2}. 

PROOF. Observe that 

n 

N - N1{ N > 2 } = !{at=i} < !{N>i} < N = ^2 l{tfc«»}- 

k=i 
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The first result follows by taking expectations. 
For the second result, note that 

n n 

(19) Yl r l - ( N + 1)1{*>2} <r i <J24 + (N + 1) v> 2} . 

k=l k=l 

Indeed, on {N < 1}, (19) holds with equality and on {N > 2}, the left-hand 
side is at most —1, whereas the right-hand side is at least +1. The result 
follows by taking expectations. □ 

8.2. Heavy and light nodes. 

Definition 8.3. Let a = (x,y) be a pair of stationary strategies and 
let 5 > 0. A node s 6 Sq is 5-heavy with respect to a if P a (t < oo\F s ) > 5; 
that is, the probability of termination in the first round given that the 
chosen branch passes through s is at least 5. The node s is 5 -light w.r.t. a 
if P a (t < oo\F s ) <S. 

For a fixed 5, we denote by H$(o) the set of 5-heavy nodes w.r.t. a. Two 
simple implications of this definition follow: 

Fact 1. We have H s (a{) C H s (ai + a 2 ). 

Fact 2. We have Hs 1 (a) C Hs 2 (a) whenever 5i>5 2 . 

The following lemma asserts that for every e-equilibrium (x, y) with high 
payoffs, H e (x,y) is nonempty, regardless of the size of the tree. 

Lemma 8.4. Let eg (0, 1/(36A') 2 ) and let (x,y) be a stationary s-equi- 
librium such that K 1 — e <j l (x,y), i = l,2. Then H e (x,y) ^ 4>. In particular, 
by Fact 2, H £ 2(x,y) ^<j>. 

Comment. The proof hinges on the assumption that -R^n s < R 2 when- 
ever -Rjx} s = R ■ As a counterexample when this condition does not hold, 

take a game in which (a) Rq s = 1 for every i, Q and s, and (b) R — R — 1. 
Then any stationary strategy pair which stops with positive probability is a 
0-equilibrium. 

Proof of Lemma 8.4. We prove that there is a node s such that 
x s y s > e, so that s € H £ (x,y). The idea is that if H £ (x,y) is empty, then the 
probability that both players stop simultaneously under (x, y) is small, but 
if (x,y) is an e-equilibrium, this probability must be high. 
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Denote tt\ = P x ,y(t < oo, Q = {1}), 7r 2 = ^x,y(t < 00, Q = {2}) and 71*1,2 = 
T?x,y(t < 00, Q = {1,2}). These are the probabilities that in a single round, 
player 1 stops alone, player 2 stops alone and both players stop simultane- 
ously, respectively. 

Since for some player i, R l is strictly positive, and hence strictly larger 
than e, we have 7*(x, y) > R l — e > 0. Therefore, 7ri + 7r 2 + 71-1,2 > 0. Assume 

W.l.O.g. that 7Ti > 7T 2 . 

Suppose to the contrary that x s y s < e for every node s £ So. The proba- 
bility that both players stop simultaneously at s, conditional on the game 
terminating at s, is 

J{x s ,ys) 



x s (l - y s ) + (1 - x s )y s + x s y s ' 



The maximum of the function f(x s ,y s ) over the set {x s y s < e, < x s , y s < 1} 
is attained at the point x s = y s = \fe. Therefore, f(x s ,y s ) < f(y/e, \/e) < 
\fe. This implies that 71*1,2 < yfe. 

Since (x,y) is an e-equilibrium and since payoffs are bounded by 1, 



i? 2 -£<7 2 (*,2/) 



^ 7Ti max sgSo R{i} )S + 7T2 max seS „ R{ 2 }, s + n ^ max se s ^{1,2},. 

7Tl + 7T2 + 71*1,2 

•Ti(fl 2 -l/iO+7r 2 fl 2 + 7ri |2 

7Tl + 7T2 + 7Tl 9 



< 



Since 7t*i > 7T2 and e < 1/4K, this implies that 71*1,2 > 1/6K. In particular, 
y/s > tti,2 > 1/6K, which is a contradiction when e < 1/36K 2 . □ 

8.3. Orthogonal strategies. 

Definition 8.5. Let 5 > 0. A sequence (ai,a2, • • • ,cx n ) of stationary 
strategy pairs is 5-orthogonal if ak+i(s) = (0,0) for every 1 < < n — 1 and 
every node s E H$(a\ + ••• that is, a^+i continues on 5-heavy nodes 

of ai + • • • + afc. 

Lemma 8.6. Lei 5 > 0, let (a±, . . . ,a n ) be a 5-orthogonal sequence of 
stationary strategy pairs, let k & {1, . . . , n} and let s £ S be a node of depth j. 
Then 



(20) p ({j < t k < 00} n ^ |J {*, < °°}) 



F s <6xP(j<t k <oo\F s ). 



Proof. Fix k £ {l,...,n}. We prove the lemma by induction on the 
nodes of T, starting from the leaves and climbing up to the root. 
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Let s e Si be a leaf of T. Since s is a leaf, P(j <t k < oo) = and (20) is 
trivially satisfied. 

Assume now that s £ So. Then 

p (^{j < t k < oo} n ^jj Oi < oo}^J \f)j 

(21) =?({tk=j}n(^J{t l <o ^\F a J 

+ Psl s '} xp({i + i<t fe <oo}n ( |J{^<oo}J|f s / J. 

By the induction hypothesis, for every child s' £ C s , 



P + l < t k < oo}n < oo}^) |F s /j < 5 



;xP(i + l<t fc <oo|F s , 

V \«</c / 1 / 

(22) 

By Lemma 8.1, {t k = j} and {Ji <k {ti < °°} are independent given F s . There- 
fore 

p({t k =j}n(\J{t l <^})\FA =P(t k =j\F s )xp(\J{t l <oo}\F s ). 

If s is 5-light w.r.t. ai+ ■■■ -j-afc_i, then P(|J; <fc {t; < oo}|F s ) < <5. If s is 
5-heavy, then, according to the definition of orthogonality, P(t k = j\F s ) = 0. 
Therefore, 

(23) p({i fe =i}n ^Uta <0 °})| F «J <<*xP(t fe =j|F s ). 

Equations (21)-(23) yield 



P y{j < t k < oo} n ^jj {t, < oo}^) ^ 



<<5xP(t fc =j|F fl ) + <Jx ^ Ps [s'} xP(j + l<t k <(x\F s ,) 

s'ec a 

= 5xP(j<t k <oo\F s ), 
as desired. □ 



Applying Lemma 8.6 to the root we get: 
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Corollary 8.7. Let 5 > and let (a±, . . . ,a n ) be a 5-orthogonal se- 
quence of stationary strategy pairs. For every k E {1, . . . ,n}, 

P^{i fc <oo}n ^JJ{tj<oo}^ <5xP({t k <oc})=5ir k . 

Lemma 8.8. Let5>0,let( a.\ , . . . , o. n ) be a 5-orthogonal sequence of sta- 
tionary strategy pairs and let N = J2k=i l{i fc <oo}- Then E[(iV + l)l/jy> 2 }] < 
35(7ri + 7r 2 H hvr n ). 

Proof. Observe that N+l < 3(N-1) on {N > 2} and (N - l)l {A r> 2 } = 
£2=1 1 {t k <oo}n([j l<k {t l <oo}y Therefore, 

E[(N + 1)1 {N > 2} ] < 3E[(N - 1)1 {N > 2} ] 

= 3±p({t k <oc}n([j{t l <oc})). 

k=l \ \l<k ) ) 

The result follows by Corollary 8.7. □ 

From Lemmas 8.2 and 8.8 we get the following: 

Corollary 8.9. Let 5 > and let (ai,...,a n ) be a 5-orthogonal se- 
quence of strategy pairs. Denote a = a\ + • • • + a n . Then for i = 1, 2: 

1. We to (1 - 35) Efe=i vr fc < vr < £fc=i vr fc . 

2. have ELi Pi " 35 ELi ^ < ^ < ELi 4 + 35 ELi ^ • 

Lemma 8.10. Le£ 5 > and Ze£ (ai, . . . ,a n ) be a 5-orthogonal sequence 
of stationary strategy pairs. Denote a = ct\ + • • • +ct n . Then for z = 1,2, 

n n n n n 

k=l k=l k=l k=l k=l 

PROOF. By Corollary 8.9 and (6), 

n 



Y,p l k-^Y,^<p i =l l X7T< 



k=l 



k=l k=l 



(i-3«5)X)7r fcJ if - 1 < y < 0. 



7 

k=l 



In both cases, the right-hand side is bounded by 7* x J2k=i ^k + 35J2k=i ^k-, 
so that 

n n n 

pk - 65 Yl < y x J2 vrfc. 
/c=i k=i k=i 
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The proof of the right-hand inequality is similar. □ 
From Lemma 8.10 and (6) we get: 

Corollary 8.11. Let 5 > and let (a\, . . . ,a n ) be a 5-orthogonal se- 
quence of stationary strategy pairs. Denote a = a\ + ■ ■ ■ + a n . Let — 1 < 
u,v < 1. 

1. // u < for each k G {1, . . . , n}, then u — 65 < j 1 . 
2- If 7/L < v for each k G {1, . . . , n}, then 7* < v + 65. 

8.4. Strong orthogonality. In the present section we define a stronger 
notion of orthogonality and study its properties. 

Definition 8.12. Let 5 > 0. A sequence (ai,ct2, ■ ■ ■ ,a n ) of stationary 
strategy pairs is 5-strongly orthogonal if, for every k G {1, ...,n — 1} and 
every node s G Hs(a\ + ■•■ +otk), ak+i(s') = (0,0) for s' = s and for every 
descendent s' of s; that is, otk+i continues from s onward. 

The following lemma provides a way to construct e-orthogonal sequences 
of strategy pairs from a single e 2 -strongly orthogonal sequence. 

Lemma 8.13. Let e > and let yi,y2, ■ ■ ■ ,Un be stationary strategies of 
player 2 such that the sequence ((0,yi), . . . , (0,y n )) is e 2 -strongly orthogonal. 
Let x be any pure stationary strategy of player 1 that does not stop twice on 
the same branch; that is, if x{s) = 1, then x(s') = for every descendant s' 
of s. Define strategies (xfc)^ =1 of player 1 in the following way: For each 
s G S such that x(s) = 1, let x^s) = 1, where k < n is the greatest index 
for which s ^ H £ ((0, y\) + ••• + (0,yk~i)) ■ Define Xk(s) =0 otherwise. Let 
Ofc = (xkiUk)- Then the sequence (cci, . . . , a n ) is e-orthogonal. 

Proof. By the definition of {xk)\<k<n arid Fact 1, we get, for every 
ZG{l,...,n-l}: 

If xi(s) = 1 then s G H e ((0, Vl ) + • ■ • + (0, y z )); 

(24) 

If xi +1 (s) = 1 then s f H s ((0, Vl ) + • ■ • + (0, yi)). 

Let I G {1, . . . ,n — 1} and let s G S be e-heavy with respect to 07 = a\ + • • • + a;. 
We prove that x/ + i(s) = yz + i(s) = 0. 

We first prove that x~i + i(s) = 0. Since s is e-heavy w.r.t. cj; = «i + • • • +<5;, 
PCT,(i < oo|F s ) > e. Assume to the contrary that xi + ±(s) = 1. By (24), s is 
e-light w.r.t. (0,yi)+ • •• +(0,y;) and, therefore, P ( o i2/1 ) + ... + (o >2/! )(i < c>o|F s ) < 
e. It follows that P ( - ii )_j ^(x lt o)(^ < 00 1 -^s) > 0, a contradiction to the as- 
sumption that x does not stop twice on the same branch. 
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We proceed to prove that y;+i(s) = 0. Assume first that there exists an 
ancestor s' of s such that xi(s') + • • • + xi(s') = 1. By (24) and Fact 1, s' G 
He((0,yi)^ K°>2//))- Since ((°>yi)j---)(° 7 yn)) is e-strongly orthogonal, 

yi+i{s) = o. 

We assume now that xi(s') + • • • + xi(s') = for every ancestor s' of s. 
Let D be the (possibly empty) set of s's descendants d that are e-heavy 
w.r.t. (0, yi) + • • • + (0, y/) and let D be the set that is obtained by removing 
from D all nodes that have strict ancestors in D. By the definition of D, 
P(p > y 1 ) + - + (0,yi)( t < °°\ F d) > £ for every d G D. Let Y = {JdeD F d- Since this 
is a mutually disjoint union, it follows that if Y ^ (f>, then 

P ( o, yi) + - + (o^)( t < °°l y ) ^^ xP ^< 
By (24) and the definition of (xk)i<k<n, it follows that 

P ( o,j /1 )+-+(o,y i )( t < ™\ YCnF *) = P ^(* < oo|y c n f s ) 

>exP^ (t<oo|y c nF s ). 

Combining the last two inequalities and observing that Y <ZF S , we get 

p (o llft ) + ... + (o lM )(* < oo|^.) > e x Pg,(i < oo|F s ) > e 2 . 

Thus s is e 2 -heavy with respect to (0, yi) + • • ■ 4- (0, y;) and, as the sequence 
((0,yi), . . . , (0,y n )) is e 2 -orthogonal, yj +1 (s) = 0. □ 

Lemma 8.14. Let e G (0, g) and Ze£ ai, a 2 G [—1, 1]. £e£ (ai, . . . , a n ) &e 
an e 2 -strongly orthogonal sequence of stationary strategy pairs such that ak is 
an e- equilibrium for each k = 1, . . . , n. Assume that for each k, 7^ G [tti,Oi + 
e] x [02 , a2 + e] , where 7^ is the payoff that corresponds to . Let a = 
ol\ -f- • • • + a n = (2, y) . Then: 

(a) We have ai — e < rf(x,y) . 

(b) For eac/i pair (x',y') of stationary strategies, 7 1 (x',y) < ai + 8e and 
7 2 (x,y') < a 2 + 8e. 

Proof. Denote a& = (xk,Vk)- We prove the result only for player 1. 
We first prove (a). Since a± < ^/l(xk,yk) for each 1 < k < n, it follows from 
Corollary 8.11 and since e < 1/6, that a\ — e < a\ — 6e 2 < 7 1 (x,y). 

We now prove (b). Let a; be a stationary strategy that maximizes player 
l's payoff against y: 7 1 (:r,y) = max x / 7 1 (x', y). Fixing y, the game reduces 
to a Markov decision process and hence such an x exists. Moreover, there 
exists such a strategy x that is pure [i.e., x(s) G {0, 1} for every s] and stops 
at most once in every branch. Observe that since the sequence (a±, . . . , a n ) is 
e 2 -strongly orthogonal, so is the sequence ((0,yi), . . . , (0,y n )). Let 27, . . .,Xf. 
be the strategies defined in Lemma 8.13 w.r.t. x and yi, . . . ,y n - Then x = 
x\ + • • • + x n and (07, . .. ,a n ) is e-orthogonal, where ay. = (xk>Uk)- 
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For each k, (xk,yk) is an e-equilibrium and, therefore, 7 1 (x/ c , yk) < a\ + 2e. 
By Corollary 8.11 and the definition of x, for every x' we have 7 1 (x',y) < 
-y 1 (x,y)<ai + 2e + 6e = ai+8e. □ 

8.5. Proof of Proposition 5.5. We now prove Proposition 5.5. Consider 
the following recursive procedure: 

1. Initialization: Start with the game T = T, the strategy pair do = (0,0) 
(always continue) and k = 0. 

2. If there exists a stationary e-equilibrium in a subgame T' of T with 
corresponding payoff in [ai,a\ + e] x [02, 02 + e]: 

(a) Set /c = k + 1 and let = (xk,yk) be any such e-equilibrium. Extend 
Xk and yk to strategies on T by setting Xk(s) = yk(s) = for every 
node s £ So \ T' . 

(b) Set a k = a k -i + a k . 

(c) Let Hk = H £ 2(ak) be the set of e 2 -heavy nodes of ak (by Fact 1, 
Hk^iQH k ). Setf = T Hk . 

(d) Start stage 2 all over. 

3. If, for all subgames T' of T, there are no e-equilibria in T' with corre- 
sponding payoff in [a\, a± + e] x [02,02 + e], set n = k, x = X\ 4- • • • +x n , 
y = yi -i — + yn and D = H n . 

The idea is to keep adding strongly orthogonal e-equilibria as long as we 
can. The procedure continues until there is no e-equilibrium in any subgame 
of T with payoffs in [a± , a\ +s] x [02 , CL2 + £ ] ■ The termination of the procedure 
follows from Lemma 8.4. 

The first part of Proposition 5.5 is an immediate consequence of the termi- 
nation of the procedure. We now prove that a n = (x,y) satisfies the require- 
ments of the second part. Since D = H n is the set of e 2 -heavy nodes of (x, y), 
n(x,y) > e 2 ><PD- For every 1 < k < n, Y( x k,yk) >R l -£, so that (x k ,yk) is 
an e-equilibrium in T. Thus ((xi,yi), . . . , (x n ,y n )) is an e 2 -strongly orthog- 
onal sequence of stationary e-equilibria. The remaining claims of Proposi- 
tion 5.5 follow from Lemma 8.14. 

9. More than two players. When there are more than two players, it 
is no longer true that games on a tree admit stationary e-equilibria. An 
example of a three-player game where this phenomenon happens was first 
found by Flesch, Thuijsman and Vrieze (1997). Nevertheless, a consequence 
of Solan (1999) is that every three-player game on a tree admits a periodic e- 
equilibrium, but the period may be long [see Solan (2001)]. We do not know 
whether this result can be used to generalize Proposition 5.5 for three-player 
games. 

When there are at least four players, the existence of e-equilibria in stop- 
ping games on finite trees is still an open problem, even in the deterministic 
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case; that is, when every node in the tree has at most a single child. For 
more details, the reader is referred to Solan and Vieille (2001). 

Acknowledgment. We thank an anonymous referee for helpful comments, 
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